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ABSTRACT 



The recognition pattern device is intended to improve the 
recognition ratio. In a pre-processing unit 3, an input pattern 
is prepared based on the image taken from a video camera 
1. Then, in a comparing processing unit 4, the input pattern 
is compared with a basic pattern stored in a function learning 
storing unit, and the deformed amount of the input pattern to 
the basic pattern is calculated. Thus, in a deformed amount 
analysis unit, the deformed amount is analyzed. Rnally, in 
a person* s information learning storing unit, on the basis of 
the above result, the standard pattern stored therein is 
regenerated. 

6 Claims, 10 Drawing Sheets 
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PATTERN RECOGNITION DEVICE 

BACKGROUND OF THE INVENTION 

1. Field of the Invention ^ 
The present invention relates to a paltetn recognition 

device suitable for image recognition or the like. 

2. Description of the Related Art 

FIG. 8 is a block diagram showing ihe construction of one lo 
example of prior an image recognition devices. A luminance 
level I (x, y) on the xy plane as an image data, for example, 
a penon's face image photographed by a video camera (not 
shown) or the like is inputted in a pre-processing unit 21. In 
the pre-processing unit 21, the characteristic amount of the 15 
image data I (x, y), for example, an image edge P (x, y) is 
detected, and is outpuited to an analysis unit 22. 

The analysis unit 22 performs a main component analysis 
or the like for the characteristic amount P (x, y) of the 
person's image outputted from the pre-processing unit 21, It 20 
calculates a contribution degree X, of the characteristic 
amount P (x, y) of the person's image, for example, to each 
of functions F, (x, y) (i=l, 2, . . , , r) of r pieces previously 
stored in a function storing unit 23, and outputs it to a pattern 
classifying unit 24. 25 

The pattern classifying unit 24, when the device is in a 
learning mode, stores the contribution degree of the 
characteristic amount P (x, y) of the person*s image output- 
ted by the analysis unit 22 in a memory (not shown) 
contained therein, in correspondence to the person informa- 
tion K (t) being the function of, for example, the number t 
given to the person (t=l, 2, . . . , T: T is Ac number of the 
person's faces) as the recognition result. In this case, for 
example, an average value of a plurality of contribution 
degrees X^, X/, X/', X/", ... for the image of the same 
person t is taken as the person information K (t). 

The pattern classifying unit 24, when the device is in a 
recognition mode, calculates the Euclidean distance between 
the contribution degree X,- of the characteristic amount P (x, 
y) of the person's image outputted from the analysis unit 22, 
and a known person's information K (I) previously stored in 
the memory contained therein. It outputs the number i in the 
person's information K (t) of minimizing the distances as the 
recognition result 

The recognition of the person's face image is thus per- 
formed. 

As the method of recognizing a person's face, there has 
been known a technique using an image compression 
method called Model-Based Coding ("Treatment of Lumi- 50 
nance/Chrominance and Motion Information Applied to 3-D 
Model-based Coding of Moving Facial Images": Journal of 
Institute of Television. Vol. 45, No. 10. pl277-1287 (1991)]. 
Further, related techniques have been disclosed in the fol- 
lowing documents: ['*Eigenfaces for Recognition": Journal 55 
of Cognitive Neuroscience Vol. 3, No, 1 P71-86 (1991)] 
[CARICATURE GENERATOR: THE DYNAMICS EXAG- 
GERATION OF FACES BY COMPUTER. Susan E. Bren- 
nan in Leonardo, Vol 18, No. 3, pages 170-178; 1985], and 
[FACE TO FACE: ITS THE EXPRESSION THAT BEARS 60 
THE MESSAGE. Jeanne McDennolt in Smithsonian, Vol. 
16, No. 12, pages 112-123; March, 1986]. In tiie Model- 
Based Coding, on the coding side, as shown in FIG. 9, the 
so-called wire frame model is made to correspond to the 
person's face inputted, and the difference information (char- 65 
acteristics of the person's face to the model) is taken out and 
transmitted. On the other hand, on the decoding side, the 



35 



same model as used on the coding side is deformed on the 
basis of the above difference information, to reproduce the 
person's face. 

Accordingly, in recognition of the person's face using the 
Model-Based-Coding, the diffwencc information between 
the inputted image of the person's face (FIG. lOa) and the 
model (FIG. 10b) is first taken. 

Namely, the person' s face image (FIG. 10a) photographed 
by a video camera is mputted, for example, in a computer 
and is displayed on a CRT. Then, tiie positions of the 
person's face image displayed on the CRT (indicated at X 
marks in FIG. 10c) in correspondence to specified positions 
previously set on the wire firame model (HG. IQb), for 
example, eyes, both ends of a mouth and the like (indicated 
at X-marks in HG. 10b) are designated, for example, by 
positioning a mouse controlled cursor and "cUcking" with 
the mouse. The wire frame model is deformed as shown in 
FIG. lOd such that the positions (FIG. 10c) designated on tiie 
the person's face image are overlapped on the specified 
positions (FIG. lOfo) previously set on the wire frame model. 
Thus, the deformed amount is taken out as die difference 
information. 

This difference information thus taken out is made to 
correspond to the person's information, which is stored in a 
memory contained in the computer as the recognition infor- 
mation for that person, i.e. as the identity information. 

In recognizing a person's face, the recognition informa- 
tion most analogous to the difference information obtained 
from the inputted image of the person's face is detected, and 
the personal identity information in correspondence to the 
recognition information is outputted as the recognition 
result. 

However, in the image recognition described above, since 
the person's face is photographed by a video camera, there 
is a tendency that a vertical or horizontal deviation and a 
tilting are generated on the screen, and further, the magni- 
tudes thereof are different from each otiier. 

Accordingly, in this case, for example, in the analysis unit 
22 of FIG. 8, not only the information on the person's face 
image, but also the infonnation on the vertical or horizontal 
deviation and the positional deviation due to rotation with 
respect to the person's face image on the screen, and further 
the deviation in magnitude due to the enlargement/reduction 
ratio of a video camera, that is, the urmecessary information 
is subjected to the maiii component analysis. This brings 
about such a disadvantage as to deteriorate the recognition 
ratio. 

Further, the model as shown in FIG. lOfc must be prepared 
for each recognition object. Namely, for recognition of the 
person's face, the person's face model must be prepared, and 
for recognition of the person's hand, the person's hand 
model must be prepared. Additionally, for example, in the 
case that all the models are prepared and stored, a lot of 
memories must be prepared, thus causing a disadvantage of 
enlarging the size of the device. 

On the otiier hand, in recognition of the person's face 
using the Model-Based Coding described above, the posi- 
tions of the person's face image displayed on die CRT 
(indicated at X-marks in HG. lOc) must be manually 
selected witii a mouse, which brings about an inconve- 
nience. 

SUMMARY OF THE INVENTION 

In view of the above situations, the present invention has 
been made, and an object of the present invention is to 
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miniaiurize ihe device and to improve the recognitioii ratio, 
A pattern recognition device defined in claim 1 comprises: 
a function learning storing unit 5 as a basic pattern storing 
means for storing a basic pattern such as a function F,; a 
pre-processing unit 3 as a preparing means for preparing an 5 
input pattern P (x, y) from the inputted information such as 
an image data I (x, y); a comparing processing unit 4 as a 
comparing means for comparing the input pattern P (x, y) 
prepared by the pre-processing unit 3 with the basic pattern 
F,- (x, y) stored in the function learning storing unit 5, and for jq 
calculating a deformed amount M (x, y) of the input pattern 
(x,y) to the basic pattern F,. (x, y); a program processing step 
S14 as a deforming means for deforming the basic pattern F, 
(x, y) stored in the function learning storing unit S or the 
input pattern P (x, y) prepared by the pre-processing unit 3 15 
on the basis of the deformed amount M (x, y) outputted from 
the comparing processing unit 4; and a program processing 
step SIS as a basic pattern regenerating means for regener- 
ating the basic pattern F,- (x, y) stored in the function learning 
storing unit 5 on the basis of the basic pattern F,. (x, y) and 20 
the input pattern P (x, y) deformed by the program process- 
ing step S14. 

A pattern recognition device defined in claim 2 comprises: 
a function learning storing unit 5 as a basic pattern storing 
means for storing a basic pattem such as a function F^ (x, y); 25 
a person's information learning storing unit 7 as a standard 
pattem storing means for storing a standard pattem; a 
pre-processing unit 3 as a preparing means for preparing an 
input pattem P (x, y) from the inputted information such as 
an image data I (x, y); a comparing processing unit 4 as a 30 
comparing means for comparing the input pattem P (x, y) 
prepared by the pre-processing unit 3 with a basic pattem F, 
(x, y) stored in the function learning storing unit 5, and for 
calculating at least a deformed amount M (x, y) of the input 
pattern P (x, y) to the basic pattem F, (x, y) and, for example, 35 
a correlation amount such as a contribution ratio X^; a 
deformed amount analysis unit 6 as an analyzing means for 
analyzing the deformed amount M (x, y) calculated by the 
comparing processing unit 4; and a program processing 
steps S31 to S35 as a standard pattem regenerating means 40 
for regenerating a standard pattem stored in the person's 
information learning storing unit 7 on the basis of at least an 
analysis result Mtdr (x, y) from the deformed amount 
analysis unit 6 among the analysis result Mtdr (x, y) and the 
contribution ratio calculated by the comparing processing 45 
unit 4. 

In a pattem recognition device defined in claims 3 and 4, 
the pre-processing unit 3 filters the image data I (x, y) with 
a LOG (Laplacian Of Gaussian) filter, to detect the zero 
crossing point, and filters it with a low pass filter. 50 

In a pattem recognition device defined in claim 5, the 
program processing step S15 deforms the basic pattem f^^Ax 
(x, y), of giving the maximum contribution degree in 
the contribution degree Xi of the input pattern P (x, y), to the 
basic pattem F, (x, y). 

In a pattem recognition device defined in claims 6 and 7, 
the comparing processing unit 4 matches the input pattem P 
(x, y) with the basic pattem F,. (x, y) for each block, and 
calculates the movement amount of the block as the 
deformed amount M (x, y). 

In a pattern recognition device, preferably, the person's 
information learning storing unit 7 is constituted of a neural 
network. 

In a pattem recognition device, preferably, the program 65 
processing steps S31 to S3S regenerates the weighting factor 
of the neural network in the person's information learning 



4 

storing unit 7 on the basis of an error inverse propagation 
method. 

In a pattem recognition device defined in claims 8 and 9, 
the pre-processing unit 3 prepares the input pattern I (x, y) 
on the basis of the face image. 

In the pattern recognition device according to the present 
invention, an input pattem P (x, y) is prepared on the basis 
of an image data I (x, y). The input pattem P (x, y) is 
compared with a basic pattem F, (x, y) stored in a function 
learning storing unit 5, to calculate a deformed amount M (x, 
y) of the input pattem P (x, y) to the basic pattem F, (x, y). 

On the basis of the deformed amount M (x, y), the basic 
pattem F,- (x, y) stored in the function learning storing unit 
5 or the input pattem P (x, y) prepared by the pre-processing 
unit 3 is deformed. Thus, the basic pattern F/ (x, y) stored in 
the function learning storing unit 5 is regenerated on the 
basis of the deformed basic pattem F, (x, y) and the input 
pattem P (x, y). Accordingly, since the basic pattern F,. (x, y) 
is regenerated so as to be analogous to the input pattem P (x, 
y), the basic pattem F, (x, y) is not required to be prepared 
for each recognition object, Tliis makes it possible to reduce 
the memory capacity of the function learning storing unit 5 
for storing the basic pattem F,. (x, y), and hence to make 
small the size of the device. Further, the recognition ratio can 
be improved. 

Additionally, in the pattem recognition device of the 
present invention, the input pattem P (x, y) is compared with 
the basic pattem F, (x, y) stored in the function learning 
storing unit 5, to calculate the deformed amount M (x, y) of 
the input pattem P (x, y) to the basic pattem F, (x, y). The 
deformed amount M (x, y) is then analyzed, and the parallel 
movement component, rotational movement component and 
the enlargement/reduction component of the input pattem P 
(x, y) contained in the deformed amount M (x, y) are 
removed. Thus, on the basis of a new deformed amount Mtdr 
(x, y), a standard pattem stored in the person's information 
learning storing unit 7 is regenerated. Accordingly, it is 
possible to improve the recognition ratio. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram showing the construction of one 
embodiment of image recognition apparatuses to which a 
pattem recognition device of the present invention is 
applied; 

FIG. 2 is a flow chart for explaining the action of a 
pre-processing unit 3 of the embodiment in FIG. 1; 

FIGS. 3(fl) and 3{b) are a view for explaining a method for 
calculating a deformed amount M (x, y) in a comparing 
processing unit of the embodiment in FIG. 1; 

FIG. 4 is a flow chart for explaining the action of a 
function learning storing unit of the embodiment in FIG. 1; 

FIGS. 5(fl) and S{b) are a view showing an input pattem 
P (x, y) and a function Ff (x, y) deformed in the function 
learning storing unit in FIG. 1; 

FIG. 6 is a flow chart for explaining the action of a 
deformed amount analysis unit of the embodiment in FIG. 1; 

FIG. 7 is a flow chart for explaining the action of a 
person's information learning storing unit of the embodi- 
ment in FIG. 1; 

FIG. 8 is a block diagram showing the construction of one 
example of prior art image recognition devices; 

FIG. 9 is a view showing a wire frame model; and 

FIGS. lO(fl) to lOid) are views for explaining a method 
recognizing the person's face by Model-Based Coding. 
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DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS 

Hereinafter, embodiments of the present invention will be 
described in detail with reference to the drawings. ^ 

FIG. 1 is a block diagram showing the construction of one 
embodiment of image recognition apparatuses to which a 
pattern recognition device of the present invention is 
applied. A video camera 1 has a CCD, which converts a light 
used for photographing a person's face or the like into a face lo 
image signal as an electric signal. A memory unit 2 is 
constituted of an RAM and an A/D converter (not shown), 
which quantizes the face image signal outputted from the 
video camera 1, for example, in eight bits by means of the 
A/D convuter, and temporarily stores digital signals (face 15 
image data) such as two-dimensional luminance information 
I (x, y) on the xy plane in the RAM for each frame. 

A pre-processing unit 3 performs, for example, the edge 
detection for the face image signal I (x, y) stored in the 
memory unit 2, and takes out an input pattern P (x, y) as the 20 
characteristic amount of the face image [face image data I 
(x, y)), and outputs it into a comparing processing unit 4. 

The comparing processing imit 4 calculates the basic 
model of the characteristic amount P (x, y) of the face image 
data I (x, y) stored in a function learning staring unit 5, for ^ 
example, a contribution degree X, as a correlation amount of 
the input pattern P (x, y) of the face image data I (x, y) 
outputted from the pre-processing unit 3, for example, to 
each of functions (x, y) (i=l, 2, 3 . . . , r) of r pieces. The 
unit 4 detects the maximum contribution degree X^ax ^ 
maximum value in the contribution degree X„ and further, 
calculates a deformed amount M (x, y) as the difference 
information between a function F^^^ (x, y) of giving the 
maximum contribution degree (MAX is one of the 
numbers of 1 to r) and the input pattern P (x, y). It supplies 35 
the deformed amount M (x, y) to the function learning 
storing unit 5 and a deformed analysis unit 6. 

The function learning storing unit 5 is constituted of, for 
example, a neural network. It stores the functions F; (x, y) 
(i=l, 2, 3 .... r) of r pieces as the basic model of the ^ 
characteristic amotmt P (x, y) of the face image data I (x, y). 

Further, the function learning storing unit 5 deforms either 
the function ^^max (x, y), of giving the maximimi contribu- 
tion degree Xj^j^ detected by the comparing processing unit 
4, or the inputted pattern P (x, y) using the deformed amount 
M (x, y) calculated in the comparing processing unit 4. Thus, 
as a function of the deformed function Fji^^' and the 
deformed input pattern P (x, y) on the xy plane, the urut 5 
regenerates the function Fjyr^ (x, y) stored therein. 

The deformed amount analysis unit 6 analyzes the 
deformed amotmt M (x, y) calculated by the comparing 
processing unit 4. TTius, the imit 6 removes the components 
of the person's face image taken by the video camera 1 as 
the input pattern P (x, y) with respect to the vertical or 55 
horizontal deviation on the screen and the positional devia- 
tion due to rotation or the difiference in magnitude due to the 
enlargement/reduction ratio of the video camera 1, which are 
contained in the deformed amount M (x, y). It outputs a new 
deformed amount Mtdr (x. y) to a person's information 50 
learning storing tmit 7. 

The person's information learning storing unit 7, when 
the device is in a learning mode, stores the new deformed 
amount Mtdr (x, y) outputted from the deformed amount 
analysis unit 6 in a memory (not sho^Yn) contained therein 6S 
in correspondence to the person* s information K (t) being 
the function of the number t given to the person (face) (t=l , 



2, . . . , T: whCTC T is the nimiber of the images of a person's 
face) as Uic recognition result In this case, for example, an 
average value of a plurality of the deformed amounts Mtdr 
(x, y), Mtdr*(x, y), Mtdr"(x, y), Mtdr"' (x. y), . . . in the face 
image of the same person t is talcen as the person information 
K (I). 

Namely, the person's information learning storing unit 7, 
when the device is in the learning mode, stores the deformed 
amount Mtdr (x, y) itself of a person t outputted from the 
deformed amount analysis unit 6 as the person's informa- 
tion. Further, each time the deformed amount Mtdr (x, y) of 
the same person t is inputted, the imit 7 regenerates the 
person's information K (t) on the basis of the deformed 
amount Mtdr (x, y). 

Further, the person's information learning storing unit 7, 
when the device is in a recognition mode, calculates the 
Euclidean distance between the deformed amount Mtdr (x, 
y) outputted from the deformed amount analysis unit 6, and 
certain person information K (t) previously stored in the 
memory contained therein, and outputs tiie nimiber t in the 
person's information K (t), thereby of minimizing the dis- 
tances as the recognition result. 

The operation of the pattern recognition device of the 
present invention will be described below. When the device 
is in the learning mode, in the video camera 1, the light used 
for photographing a person's face or the like is converted 
into a face image signal as an electric signal, and is outputted 
imo a memory imit 2, In the memory unit 2, the face image 
signal (analog signal) outputted from the video camera 1 is 
quantized, for example, in eight bits in an A/D converter 
contained therein, and the two-dimensional luminance infor- 
mation I (x, y) on the xy plane as digital signals (fece image 
data) is temporarily stored in an RAM contained therein for 
each frame. 

In the pre-processing unit 3, the face image data I (x, y) 
stored in the memory unit 2 is read out, edge detection or the 
like being performed, and an input pattern P (x. y) as the 
characteristic amount of the face image [face image data I 
(x, y)] is taken out 

Namely, in the pre-processing unit 3, as shown in the flow 
chart of FIG. 2, first, in a step SI, the face image data I (x, 
y) is filtered with a LOG (laplacian Of Gaussian) filter, to 
take out the edge portion of the face image, and an edge 
signal If (x, y) is thus calculated (the edge is detected). 

Additionally, in the step SI, the edge signal (x, y) may 
be acquired by multiplying the frequency characteristic of 
the face image data I (x, y) by the frequency characteristic 
of the LOG filter. However, in this embodiment, the edge 
signal If (x, y) is acquired by two-dimensionally convolut- 
ing the face image data I (x, y) with an impulse response 
^Loo y) ^ shown in the equation of (l-l): 



In addition, cj is a specified constant set according to the 
magnitude of the LOG filter. 

The process advances to a step 2, wherein it is judged 
whether or not the product of an edge signal I^. (X;, y^) at a 
point (Xf, yj) and an edge signal I^ (x^j, yj) at a point {Xf^^, 
yj) moved from the point (x,., yJ) in the x-direction by one 
picture element is negative within a screen of the face image 
outputted from the video camera 1, that is, within the range 
of Xoix^^Xl, Yo^y^-^Yl on the xy plane. 

Here, briefly, it is assumed that the face image outputted 
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from the video camera 1 lo the pre-processing unit 3 through 
the memory unit 2 is constituted of the picture elements of 
N pieces for each of the vertical and horizontal directions. 
Further, the point (Xo. Yo) on the xy plane is taken as the 
origin (0. 0). Accordingly, it is assumed that X1=Y1=N-1. 5 

In the step S2, if the product of an edge signal (x,, y^) 
at a point (x„ y j and an edge signal (x^i, y^) at a point 
(x^j. y,) moved from the point (x,., yy) in the x-dircction by 
one picture element is judged to be negative, that is, if the 
code of the edge signal (x,., y^) at a point (x,., y^) is different 10 
from the code of the edge signal (x^i, y^) at a point (x^^, 
yJ) moved from the point (x,-, y^-) in the x-direction by one 
picture element, the process advances to a step S7, wherein 
it is judged that the zero crossing is generated between the 
point (x,-, yj) and the point (x^^^j, y^). Thus, the value of 1 15 
which designates the generation of the zero crossing is, for 
example, set in a zero crossing function (x,., y^), and the 
process advances to a step S5. 

In the step S2, if the product of the edge signal (x^, y^) 
at the point (x^ y^) and the edge signal {x^^^, y^ at the 20 
point {x,+i, y^) moved from the point (x,, y^) in the x-direc- 
tion by one picture element is judged not to be negative, the 
process advances to a step S3, wherein it is judged whether 
or not the product of an edge signal (x,, y^) at a point (x,, 
jj) and an edge signal (x,, yj^^) at a point (x„ y^^J moved 25 
from the pomt (x^ y^) in the y-direction by one picture 
element is negative. 

In the step S3, if the product of an edge signal (x,., y^) 
at a point (x,-, y^) and an edge signal I^ (x^, y^+i) at a point 



Additionally, in the step S5, the input pattern P (x. y) as 
the characteristic amount of the face image photographed by 
the video camera 1 may be acquired by multiplying the 
fipequency characteristic of the zero crossing function P^ (x, 
y) by the frequency characteristic of the Gaussian filter. 
However, in this embodiment, the input pattern P (x, y) is 
acquired by two-dimensionally convoluting the zero cross- 
ing function P^ (x, y) with an impulse response (x, y) of 
the Gaussian filter as shown in the equation of (1-2): 



i 



(1-2) 



cxp 



In addition, a is a specified constant set according to the 
magnitude of the Gaussian filter just as the LOG filter in the 
step SI. 

By the processing in the step S5, the change in the 
contribution degree X, of the input pattern P (x, y) to the 
function F^ (x, y) stored in the function learning storing unit 
5, which is detected by a comparing processing unit 4 
described later, is made smooth, thus making it possible to 
easily detect the function (x, y) of giving the maxi- 
mum contribution degree Xj^^. 

The input pattern P (x, y) calculated in the step S5 is 
outputted to the comparing processing unit 4 in the step S6, 
thus completing the processing. 

As described above, the input pattern P (x, y) as the 



characteristic amount of the face image is prepared on the 
(i;y7,Tmov/ifrom ^^T^^m^r^lJilndieTdV^Son by 30 basis of the face image data I (x, y) in the pre-processing unit 
one picture element is judged to be negative, that is, if the 



code of the edge signal l£ (x,-, y^) at a point (x,-, y^) is different 
from the code of the edge signal I^ (x,-, y^-^^j) at a point (x^, 
yj+\) moved From the point (x,-, y^) in the y-direction by one 
picture element, the process advances to the step S7, 
wherein the value of 1 is set in the zero crossing function P^ 
(x,-. y^) as described above, and the process advances to the 
step S5. 

In the step S3, if the product of the edge signal I^ (x„ y^) 
at the point (x^ y^) and the edge signal I^ (x,, y^J at the 
point (X;, y^i) moved from the point (x„ y^) in the y-direc- 
tion by one picture element is judged not to be negative, the 
process advances a step S4, wherein it is judged that the zero 
crossing is not generated between the point (x,, y^) and the 
point (x^^,, y^) or the point (x^ yj^O- Thus, the value of 0 
which designates no generation of the zero crossing is, for 
example, set in the zero crossing function P^ (x^, y^), and the 
process advances to the step S5. 

In addition, the processings from the steps 2 to 4, and the 
step 7 are performed for the point corresponding to each 
picture element within the face image screen on the xy plane 
[each point (x^, y^) in the range of 0^x,-^N-l, O^y^^N-l). 

By calculating the function P^. (x^ y^) of indicating the 
zero crossing point of the edge of the face image in the 55 
manner described above, that is. by detecting the zero 
crossing point of the edge of the face image, it is possible to 
remove the effect due to illumination or the like when the 
face image is photographed by the video camera 1. 

The process advances to the step S5, wherein the zero 60 
crossing function P^ (x, y) is filtered with a low pass filter 
such as a Gaussian filter, so that the face image pattern 
represented by the zero crossing function P^ (x. y) is 
converted into the so-called faded face image pattern, and 
the input pattern P (x, y) as the characteristic amount of the 65 
face image photographed by the video camera 1 is calcu- 
lated 



35 



40 



45 



50 



In the comparing processing unit 4, the correlation 
amount of the input pattern P (x, y) prepared in the pre- 
processing unit 3, for example, the contribution degree X,- to 
the function F,. (x, y) (i=l, 2. .... r: r is a specified number) 
as the basic pattern stored in the function learning storing 
unit 5 is calculated, and the maximum contribution degree 
^MAx ^ the maximum value is detected. 

Here, the contribution degree X^ of the input pattern P (x, 
y) to the function F^- (x, y) is the orthogonal projection of the 
input pattern P (x, y) to the function F,- (x, y), which means 
the irmer product of the function F^ (x, y) calculated accord- 
ing to the equation (2-1) and the input pattern P (x, y). 



N-\ N-l 
y=0 je=0 



(2-1) 



In addition, as described above, N is the number of the 
picture elements for each of the vertical and the horizontal 
directions of the screen in the face image outputted from the 
video camera 1 to the pre-processing unit 3 through the 
memory unit 2. 

In the comparing processing unit 4, the input pattern P (x, 
y) outputted from the pre-processing unit 3 is matched with 
the function F^^ (x, y), of giving the maximum contribu- 
tion degree X^^^ (MAX is a value from 1 to r) for each 
block, and the deformed amount M (x, y) [M' (x, y)] of the 
input pattern P (x, y) [function Fj^^ (x, y)] is calculated in 
the case that the input pattern P (x, y) (function Pj^^ (x, y)] 
is made to be most analogous to the function (x, y) 
[input pattern P (x, y)]. 

Namely, in the comparing processing unit 4, first, the 
function F^^jf (x, y) is divided into blocks FB^^ (x^^, y^) 
(k=0. 1. 2 ... , B-1) in B pieces composed of picture 
elements in b pieces for each of the vertical and horizontal 
directions, as shown in FIG. 3a. In addition, the point (x^. y^t) 
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indicates the coordinate point of the center of the block FB^ 

Next, blocks PB (x^ yy) having the center coordinate point 
0^. y,). which are composed of the picture elements of b 
pieces for each of the vertical and horizontal directions, are 5 
assumed on the input pattern P (x, y). Thus, the movement 
amount (m^^, m^J of the block FB^ (Xj^, yj is detected such 
that the block FB^ (Xjt, y^) is moved from the center point (Xj^, 
yj on the input pattern P (x, y) within the range of ±S picture 
elements in the x-direciion or the y-direciion, and is most jq 
analogous to the PB (x,, y^) on the input pattern P (x, y). 

Namely, in the comparing processing unit 4, the deformed 
amount M (x, y) is calculated (detected) as the movement 
amount (m^ nL^^) with which the contribution ratio X^^j^ 
(k) of the block PB (x^+m^jt, yg^vny^ to each block FEj^ (xj^, j5 
y^ becomes the maximum value, as shown in the equation 
(2-2): 



^AfAx ik)=<FB (Xi, y*), PB (r*+/nj*, yt+ffly^ ) 



(2-2) 



where <a, b > represents the inner product of the vectors a 20 
and b, and accordingly, 



r r ^„,^Cx,y)./'(r + injd.y + myi) 



25 



where [u] is the maximum integer not exceeding the value 

30 

Hereinafter, in the case that the block FB;^ i^k* Vi) with the 
center of die point (x^^ is most analogous to the block PB 
(X/, y^) on the input pattem P (x, y), the movement amount 
M {m^, niyj^) of the block FBj^ (Xj^ yj is represented by a 
deformed amount M (xj^, yj), and the set of the deformed 
amount M (xj^ y^) is represented by a deformed amount M 
(X. y). 

The deformed amount M (x. y) thus calculated by the 
comparing processing unit 4 is supplied to the function 
learning storing unit 5 and the deformed amount analysis 
unit 6. 40 

In the function learning storing imit 5, by use of the 
deformed amount M (x, y) calculated by the comparing 
processing unit 4, the function Tj^^ (x. y) of giving the 
maximum contribution degree Xj^^ detected by the com- 
paring processing unit 4 or the input pattem P (x, y) are 45 
deformed. Thus, the function F^^ (x, y) stored therein is 
regenerated on the basis of the deformed function ^max (x» 
y) and the deformed input pattem P' (x, y) on the xy plane. 

Namely, in the function learning storing unit 5, as shown 
in the flow chart of FIG. 4, first, in a step Sll, when the 50 
deformed amount M (x, y) [M (Xjt, y^ [=set of (in^ niyj] 
as the set of the movement amount (m^ m^J of the block 
FB* i^h y*) in the case that the block Ffij^ (x^ y^) is most 
analogous to the block PB (x„ y^) on the input pattem P (x, 
y) is inputted from the comparing processing unit 4, in a step 55 
S12, the movement amount {-m^ ~^k) of block PB (x,-, 
yj) [=PB (Xjfc+rn^ yA+niy ^ ^'o^^ 
yj) on the input pattem P (x, y) is made most analogous to 
the block FB^ (x^, yJ is calculated, to be set to a variable 
M'(Xjt+m^ yk^TOy^ indicating the movement amount (-m^^, 60 

The process advances to the step S13, wherein the set Mp 
(x, y) of a deformation active element Mp (xjt, yJ for 
deforming the input pattem P (x, y) or the function F^^^ (x, 
y), and the set Mp (X, Y) of (Xt+m^^jt. y^t+niyj are 65 
respectively calculated according to the following equations, 
and the process advances to a step S14: 



10 

A/i»Ut,y*MxAfCx;t,y») 

Mf Ui+ni^ yt+inyt>=(l^)xAr(xt+m^ . yf^^ 

where A is a constant within the range of 0^ A^l, which is 
regenerated from the small value to the large value as the 
learning of the function F, at the function learning storing 
unit 5 proceeds. 

In the step S14, assuming that the deformation active 
element M^(x, y) or (x, y) is 

M^yMdU, dly)or 

AffUy>=(d2x,d2y), 

the input pattern P (x, y) or the function F^ax (x. y) is 
deformed according to the following equation: 

PU.y)=/'U-Hilxytdly)or 

U >)=f MAX (^^^ y^yX 

Namely, the deformed input pattem F (x, y) as shown in 
FIG. 5a, and the deformed function F^^^;^' (x y) as shown in 
FIG. Sb are calculated, and the process advances to a step 
SIS. 

In the step S15, a new function F, (x, y) as the function 
^MAx (x, y) subjected to the learning according to the 
equation (2-3) is calculated on the basis of the new input 
pattem P' (x, y) and the new fanction F^^^^' (x, y), and which 
is stored in the function learning storing unit 5 in place of the 
function F^^^ (x, y), thus completing the processing. This 
new function Fj (x, y) is defined as: 



FuAxi^cy) + fXMAxP'ix.y) 



(2-3) 



where e is a specified number within the range of Cke<l . 

On the other hand, the deformed amount M (x, y) inputted 
from the comparing processing unit 4 to the deformed 
amount analysis unit 6 is analyzed thereat. Thus, the com- 
ponents of the image of the person's face photographed by 
the video camera 1 as the input pattem P (x, y) with respert 
to the vertical or horizontal deviation on the screen (parallel 
movement component), die positional deviation due to rota- 
tion (rotational movement component) or a component 
regarding the difference in the magnitude due to an enlarge- 
ment/reduction ratio of the video camera 1, which are 
contained in the deformed amount M (x, y), are removed. 
Thus, a new deformed amount Mtdr (x, y) is outputted to the 
person's information learning storing unit 7. 

Namely, in the deformed amount analysis unit 6, as shown 
in the flow chart of FIG. 6, first, in a step 21, the parallel 
movement component T contained in the input pattem P (x, 
y) is calculated by the following equation: 



I 



(3.1) 



2 I MUy) 
y=0 x=0 



Thus, the process advances to a step 22, wherein a 
deformed amount Mt (x, y) from which the parallel move- 
ment component T is removed is calculated on the basis of 
the deformed amount M (x, y) according to the following 
equation, and the process advances to a step S23. 



Mt (X, y)=A# U; y)-T 



(3-2) 



In the step S23, wherein the component D regarding the 
difference in the magnitude contained in the input pattern P 
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(x, y) (component regarding the enlargement/reduction 
ratio) is calculated according to the following equation: 



D= I I <MUy),rix;y> 
y=0. x=0 



(3-3) 



where r (x, y)={x-xO, y-yO), and (xO, yO) is the center of the 
face image, that is. (xO. yO)=(N/2, N/2). 

After that, the process advances to a step S24, wherein a 
deformed amount Mtd (x, y) obtained by removing the 
component D regarding the enlargementyreduction ratio 
from the deformed amount Mt (x, y) is calculated according 
to the following equation: 

Mtd U y)=Mt («; y}^d U y) (3-4) 

where 5Md (x, y) is defined by the following equation: 



SA/rfUy) = 



N-1 N-\ 
y=0. x=0 



)] 



R= I ZAHx.y)xrix,y) 
y=0. x=Q 



10 



20 



rU,y) 



Assuming that 5Md (x, y) is represented by the following 
equation: 

SMd {X, yy=ar (jt. y) 

(a is the number within the range of O^a^l), 
6Md (x, y) is required, by replacing the M (x, y) in th 
equation (3-3) with [M (x, y)+5Md (x, y)] such that the 
component D regarding the enlargement/reduction ratio 
becomes zero, and by replacing the component D regarding 
the enlargement/reduction ratio with zero. 

In a step S25» the rotational movement component (tilting 
component) R contained in the input pattern P (x, y) is 
calculated by the following equation: 



30 



35 



(3-5) 



40 



where M (x, y)xr (x, y) indicates the outer product of the 
vector M (x, y) and the vector r (x, y). 

The process advances to a step S26, wherein a deformed 
amount Mtdr (x, y) obtained by removing the rotational 
movement component R from the deformed amount Mtd (x, 
y) is calculated by the following equation, and the process 
advances to the step S27. 



Mtdr (I, yy=Mtd (x. yH^Mr y) (3-6) 

where 6Mr (x, y) is defined by the following equation: 



55 



60 



where s (x, y)=[-(y-yO), x-xO]. 

Assuming that 6Mr (x, y) may be represented by the 
following equation: 

6Mr U jr)=ar (r. >). 



SMr (x, y) is required by replacing the M (x, y) in th 
equation (3-5) with [M (x, y)+5Mr (x, y)] such that the 65 
rotational movement R becomes zero, and by replacing the 
rotational movement component R with zero. 



12 



As described above, the new deformed amount Mtdr (x, 
y), from which the parallel movement component T, the 
component D regarding the enlargement/reduction ratio, and 
the rotational movement component R arc removed, is 
outpuued to the person* s information learning storing unit 7 
in a step S27, thus completing the processing. 

The above processing is made for each deformed amount 
M (Xjt, yjt) (k=0, 1, . . . B-1) to each of blocks divided in B 
pieces in the function F, (x, y) in the comparing processing 
unit 4 as the constituting element of the deformed amount M 
(X. y). 

Accordingly, in the deformed amoimt analysis unit 6, the 
new deformed amount Mtdr (x^., yj to the deformed amount 
M (Xfc, yjfe) of each of blocks k (k=0, 1, . . . B-1) divided in 
B pieces in the function (x, y) (JPf^^x Y)) 
comparing processing unit 4 is calculated. 

Namely, in this specification, the set of the new deformed 
amount Mtdr (x^. yjj to the deformed amount M (xjr., yj of 
each of blocks k divided in B pieces in the function F,- (x, y) 
[Pmax (x, y)] in the comparing processing unit 4 is described 
as the new deformed amount Mtdr (x, y). 

Further, since the deformed amount Mtdr (x^^ y^) is the 
two-dimensional vector, the new deformed amount Mtdr (x, 
y) as the set of the deformed amount Mtdr (x^, y^) may be 
regarded as the 2B-dimensional vector. 

As the processing in the deformed amount analysis unit 6 
is completed, in the person's information learning storing 
unit 7, the new deformed amount Mtdr (x, y) calculated by 
the deformed amount analysis unit 6 is stored in the memory 
contained therein, in correspondence to the person's infor- 
mation (standard pattern) K (t) being the f\mction of the 
number t (t=l, 2 . . . , T: where T is the number of the 
person's face images) given to the person as the recognition 
result 

Namely, in the person's information learning storing unit 
7, as shown in the flow chart of FIG. 7, first, in a step S31, 
when the number t given to the person is inputted, the 
person's information K (t) as the standard pattern is read out 
from the memory contained in the person's information 
learning storing unit 7 in a step S32, and the process 
advances to a step S33. 

In the step S33, as the deformed amount Mtdr (x, y) is 
inputted from the deformed amount analysis unit 6 to the 
person's information learning unit 7, in a step S34, the 
person's information K (t) is regenerated on the basis of the 
deformed amount Mtdr (x, y) according to the following 
equation: 

K it:2ky=K {t:2kh-axMtdr^ (x*, y*) 



50 



K (/;2Jt+l)=/r (t:2k+l}+axMtdry Uj,, yO 

where k=0, 1, . . . , B-1. 

Here, Mtdr^ (x^^, y^^) or Mtdr^ (Xj^, y^) indicate the x-com- 
ponent or the y-component on the xy plane of the new 
deformed amount Mtdr (x^ yj in the block (FIG, 3a) of the 
function F, (x, y) with the center of the point (x^., y^). 

Further, since the new deformed amount Mtdr (x, y) is the 
2B -dimensional vector as described above, the person's 
information K (t) is the 2B-dimensiona] vector similarly. 
The K (t:2k) and K (t:2k-f-l) indicate the 2k;dimensional and 
the (2k-t-l)-dimensional elements of the person's informa- 
tion k (t), respectively. 

In addition, a is a specified constant within the range of 
0<:a<l. 

The process advances to a step S35, wherein the K(t) 
regenerated in the step S34 is stored in the memory con- 
tained in the person's information learning storing unit 7, 
thus completing the processing. 
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Next, when the device is in the recognition mode, in the 
video camera 1, the memory unit 2, the pre-processing unit 
3, the comparing processing unit 4, the function learning 
storing unit 5, or the deformed amount analysis unit 6, the 
same processing as described above is made, and the new 5 
deformed amount Mtdr (x, y) is inputted in the person's 
information learning storing unit 7. Accordingly, in the 
person's information learning storing unit 7, the Euclidean 
distance between the deformed amount Mtdr (x, y) and 
certain person's information K (t) stored in the memory 10 
contained therein is calculated, and the number t in the 
person's information K(t) of minimizing the distances is 
outputted as the recognition result 

In this embodiment, the pre-processing unit 3 filters the 
image data with the LOG filter to detect the image edge; is 
however, the detection method for the image edge is not 
limited thereto. Further, in the pre-processing unit 3, it is 
possible to take out not only the image edge but also the 
other characteristic amount. In addition, since the problem 
of correspondence to the image is solved at the comparing 20 
processing unit 4, at the pro-processing unit 3, it is possible 
to output the image data to the comparing processing unit 4 
without any filtering. 

In the comparing processing unit 4, the deformed amount 
M (x, y) is calculated by the block matching; however, the 25 
deformed amount M (x. y) can be calculated by the optical 
flow method commonly used in detection of the movement 
of the movable image as disclosed in, for example, Japanese 
Patent Laid-open No. HEI 3-150520. 

In the function learning storing unit 5, only the function 30 
^MAx O^f y) of giving the maximum contribution degree 
Xa£ax ts deformed G&anied); however, the function of giving 
the secondarily or thirdly larger contribution degree may be 
deformed Oeamed). 

The person's information learning storing unit 7 may be 35 
constituted of the neural network just as the function learn- 
ing storing unit 5, wherein the contribution degree X,. 
calculated in the comparing processing unit 4 is inputted in 
the person's information learning storing unit 7 (as shown in 
the dotted line of FIG. 1), so that the person's information 40 
K (t) can be learned according to the error inverse propa- 
gation method by use of the deformed amount Mtdr (x, y) 
and the contribution degree X,. that is, the weightening 
factor of the neural network can be regenerated. Also, by 
inputting the parallel movement component T, the compo- 45 
nent regarding the enlargement/reduction ratio or the rota- 
tional movement component R calculated by the deformed 
amount analysis unit 6 in the person's information learning 
storing urut 7. it is possible to perform the learning of the 
pM^on's information K (t). Thus, it is possible to judge the 50 
position, magnitude or the tilting of the substance (image) to 
be recognized. 

In the person's information learning storing unit 7, it is 
possible to perform tiic learning by the so-called main 
component analysis method. 55 

Further, in the function learning storing unit 5, or the 
person's learning storing unit 7, there may used, for 
example, the learning methods using the so-called Boltz- 
mann machine and the simulated annealing. 

As described above, according to the pattern recognition 60 
device of the present invention, an input patt^ from the 
information of the image is prepared, and is compared with 
the basic pattern stored in the basic pattern storing means 
and calculates the deformed amount of the input pattern to 
the basic pattern. Subsequently, on the basis of the deformed 65 
amount, the basic pattern stored in the basic pattern storing 
means and the input pattern prepared by the preparing means 
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are deformed. Thus, on die basis of the deformed basic 
pattern and the input pattern, the basic pattern stored in the 
basic pattern storing menas is regenerated. Accordingly, 
since the basic pattern is regenerated so as to be analogous 
to the input pattern, it is unnecessary to prepare the basic 
pattern for each recognition object, thus making it possible 
to reduce the storing capacity of the basic pattern storing 
means for the storing the basic pattern, and hence to make 
smaller the size of the device. Further, it is possible to 
improve the recognition ratio. 

Further, according to the pattern recognition device of the 
present invention, the input pattern is compared with the 
basic pattem stored in the basic pattern storing means, so 
that the deformed amount of the input pattem to the basic 
pattem is calculated. Then, the deformed amount is ana- 
lyzed, and on the basis of the result, the standard pattem 
stored in the standard pattem storing means is regenerated. 
Accordingly, it is possible to automatically perform the 
regeneration Qeaming) of the standard pattem so as to 
improve the recognition ratio. 

What is claimed is; 

1. A pattern recognition device comprising: 

a basic pattem storing means for storing a basic pattem; 
a standard pattem storing means for storing a standard 
pattem; 

a preparing means for preparing an input pattem on the 
basis of inputted information; 

a comparing means for comparing the input pattern pre- 
pared by the preparing means with the basic pattem 
stored in the basic pattem storing means, and calculat- 
ing at least a deformed amount of the input pattem to 
the basic pattem and a correlation amount; 

an analyzing means for analyzing the deformed amoimt 
calculated by the comparing means and generating an 
analysis result; and 

a standard pattem regenerating means for regenerating a 
standard pattem stored in the standard partem storing 
means as a function of at least the analysis result from 
the analyzing means and the correlation amount calcu- 
lated by the comparing means. 

2. A pattem recognition means according to claim 1, 
wherein the preparing means filters the inputted information 
with a Laplacean of Gaussian (LOG) filter, to detect a zero 
crossing point, and fillers the inputted information with a 
low pass filter. 

3. A pattern recognition device according to claim 1, 
wherein the basic pattem is divided into a series of blocks 
and the comparing means matches the input pattem with the 
basic pattem for each block, and calculates a movement 
amount of each block as the deformed amount 

4. A pattem recognition device according to claim 1, 
wherein the preparing means prepares the input pattern as a 
function of a face image. 

5. A pattem recognition device according to claim 1, 
wherein \hc preparing means prepare an input pattem on the 
basis of inputted video image information of a particular 
person's face, the basic pattem storing means stores image 
data of a himian face as a basic pattem, and the standard 
pattem storing means stores visual image identification data 
as a standard pattcra 

6. A pattem recognition device according to claim 5, 
wherein the standard pattcra is obtained from a plurality of 
visual images from the same person. 

♦ * ♦ ♦ * 
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